Predictive flow switching and application continuity in connected vehicle networks

ABSTRACT

The subject disclosure relates to ways to ensure vehicle network connectivity. In some aspects, a process of the technology includes steps for receiving local network measurement data including one or more connectivity metrics for at least one network provider (e.g., an ISP), and updating a geo-connectivity database using the received local network measurement data. In some aspects, the process can include additional steps for transmitting a geo-connectivity request to a remote Intelligent Vehicle Connectivity Analytics (IVCA) gateway, and for receiving a geo-connectivity reply from the remote IVCA gateway, the geo-connectivity reply including information regarding network availability for the at least one network provider along a vehicle path. Systems and machine-readable media are also provided.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.17/076,442, filed Oct. 21, 2020, entitled “Predictive Flow Switching andApplication Continuity In Connected Vehicle Networks,” which is acontinuation of U.S. application Ser. No. 15/951,050, filed Apr. 11,2018, entitled “Predictive Flow Switching and Application Continuity inConnected Vehicle Networks”, which claims priority to U.S. ProvisionalApplication No. 62/484,491, filed Apr. 12, 2017, entitled “PredictiveFlow Switching and Application Continuity in Connected VehicleNetworks”, all of which are herein incorporated by reference in theirentireties.

BACKGROUND Technical Field

The subject technology relates to systems and methods for performingintelligent flow switching across different provider networks (“providerpaths”) for a network connected vehicle.

INTRODUCTION

Wireless communication technologies are increasingly being deployed invehicles, including, but not limited to: cars, motorcycles, trains, andboats etc. Vehicle connectivity is useful for enabling a variety ofcommunication and entertainment applications, such as for supportingtelephone, navigation, and streaming media applications.

BRIEF DESCRIPTION OF THE DRAWINGS

Certain features of the subject technology are set forth in the appendedclaims. However, the accompanying drawings, which are included toprovide further understanding, illustrate disclosed aspects and togetherwith the description serve to explain the principles of the subjecttechnology. In the drawings:

FIG. 1A illustrates an example of a network connectivity environmentthat can include wired and/or wireless devices that a moving vehicle islikely to encounter, with varying degrees of connectivity or noconnectivity.

FIG. 1B conceptually illustrates an example of how network holes canappear as a vehicle moves, and how various servers and services can aidin their timely detection.

FIG. 2 illustrates an example of a Predictive Multi Homing Switch andhow it can be operatively implemented inside existing in-vehicle localarea networks (LANs) and configured to provide continuous applicationconnectivity to a computer network, such as the Internet.

FIG. 3 depicts an example of how network measurements can be crowdsourced and used to facilitate the creation of a geo-connectivity map,according to some aspects of the technology.

FIG. 4 conceptually illustrates an example of a predictive flowplacement that is performed based on the anticipated approach of anetwork hole, according to some aspects of the technology.

FIG. 5 illustrates an example process performed at least in part by aPredictive Multi-homed WAN Switch (PMHS) to compute and update a locallystored geo-connectivity database.

FIG. 6 illustrates an example of an Intelligent Vehicle ConnectivityAnalytics (IVCA) gateway operation and its use of a crowd-sourcealgorithm to collect network metrics from one or more vehicles reachablevia a network, such as the Internet or an equivalent global reachnetwork.

FIG. 7 illustrates one aspect of the invention utilizingvehicle-to-vehicle paths to solve a network hole problem resulting fromblocked network access via certain provider path types, such as viasatellite signals.

FIG. 8 illustrates one aspect of the invention used to detect roadtunnels where Long Term Evolution (LTE) signals are enabled internally,by communicating with an IVCA gateway.

FIG. 9 illustrates a block diagram of hardware components that can beused to implement a PMHS and/or IVCA of the disclosed technology.

DETAILED DESCRIPTION

The detailed description set forth below is intended as a description ofvarious configurations of the subject technology and is not intended torepresent the only configurations in which the subject technology can bepracticed. The appended drawings are incorporated herein and constitutea part of the detailed description. The detailed description includesspecific details for the purpose of providing a more thoroughunderstanding of the subject technology. However, it will be clear andapparent that the subject technology is not limited to the specificdetails set forth herein and may be practiced without these details. Insome instances, structures and components are shown in block diagramform in order to avoid obscuring the concepts of the subject technology.

1.0 Overview

As vehicles (e.g., cars, boats, ships, aero planes, etc.), become moreintelligent with multiple computing devices installed or present inthem, they also require network connectivity while on the move. In thecurrent state of the art there are various wireless network optionse.g., 4G, LTE, 5G, Satellite networks, IEEE 802.11a/b/g/n networks, andDedicated Short-Range Communications (DSRC) IEEE 802.11p networks, etc.for connecting vehicles on the move—all of which come with variousbenefits and limitations. Cities and states are now considering buildingsmart infrastructure based on V2V (vehicle-to-vehicle), V2I(vehicle-to-infrastructure), and V2P (vehicle-to-pedestrian)—and thereare several candidate technologies for these new V2Xdeployments—including DSRC and Cellular V2X based on 5G standards beingdeveloped by the Global System for Mobile Association (GSMA).

Examples of such wireless network access links 161, 162 and 163 (oraccess clouds or access paths) are illustrated in FIGS. 1A and 1B as twomoving vehicles 101 and 102 travel from one location to another on aroute 131.

Some of the driving factors for increasing computation capability invehicles are: safety, emergency calling via voice and data messages,high definition maps for navigation, infotainment and mobile office,improving fuel efficiency, Advanced Driver Assist System (ADAS), OverThe Air (OTA) software updates, Firmware Over the Air (FOTA) updates forElectronic Control Units (ECUs), and/or On Board Diagnostics (OBD), etc.These applications, which increasingly require a network connection,place a set of stringent requirements on the availability and quality ofnetwork services, in particular wireless network services. At times whenvehicles are parked and not moving, they may require a mix of wired andwireless network services. In this context, a number of problemsregarding the nature of connectivity exist that need to be overcome ormitigated. Several of these problems are described below.

1.1 Single Homing for Vehicles

The majority of vehicles today depend on a single communicationsprovider, which could be a cellular phone service provider (e.g., forcars) or a satellite service provider, e.g., for marine-based vehicles,such as ships, and some cars. Typically, the availability of thesenetwork services with adequate performance for today's in-vehiclecomputing applications is seldom better than 99.9%, which translatesinto more than 8 hours of downtime per year. This could mean on averagefour outages of two hours each per year, or eight outages of 1 houreach, per-year. In the context of a road trip that might last 30 minuteson average, this poses a serious risk that an outage happens at acritical time, such as an accident or emergency. In fixed enterprisecomputer local area networks (LANs) connected to the Internet, the needfor redundant network links and Internet multi-homing has become widelyunderstood. However, in the case of in-vehicle LANs while the vehicle ison the move, this problem of redundancy and multi homed connectivity hasnot been adequately addressed and solved.

1.2 Traditional Routing Approaches Can Break Applications

Even when multi homed wide area network (WAN) solutions have beendeployed, there are other problems. When two or more networks arepresent, each from independent service providers, another problem posesitself. In such a case, the two or more wireless based networks are runby autonomous entities, and are referred to in the network literature asAutonomous Systems (AS-es). When a failure in a specific network path(e.g., provider path) to a specific destination occurs in a first ASnetwork, application traffic for that specific destination needs to bere-routed automatically via a second AS network. The time between faultdetection on the first AS network, and a successful re-route to thesecond AS network, is called the routing convergence time. In some ASrouting protocols, e.g., the Border Gateway Protocol (BGP), thatconvergence time is often 5 minutes or longer. During this period ofconvergence, unfortunately, many applications—including voiceconversations and web applications—break. For example, with voiceapplications such as telephone calls, the speakers may hang up the callbecause there is silence and no audio signal coming in, after severalseconds of silence, simply because the speaker does not want to spendmore time listening to silence or waiting for a resumption of the voicesession. Further, as in many network devices that track the presence ofspeech traffic and detect periods of silence, there are timers thatautomatically terminate the voice session if a period of silence lastslonger than, for example, about 20 or 30 seconds. In light of theseexamples, the 5 or 10-minute convergence time for traditional routingprotocols is a serious problem, as it breaks application continuity.

1.3 Traditional Routing Protocol Solutions Are Often Impractical forSmall Networks

While BGP and similar solutions have been used in large enterprisenetworks, they are often impractical in smaller networks. Some of thedifficulties involved are: (1) the need to obtain an AS number unique inthe worldwide routing system, which is a cumbersome exercise for smallusers; (2) the need to have highly paid and competent staff toadminister the BGP router configuration, as this task is complicated andrequires expert knowledge in order to coordinate routing and frequentcollaborative troubleshooting with peering service providers' technicalexperts; (3) the monetary costs of complex router purchase andimplementation. These factors prevent conventional routing solutionsfrom scaling to support thousands or millions of small networks, as inhome networks, let alone in-vehicle networks which are embedded andtransparent to the vehicle operator.

1.4 Applications Break in Network Address Translation (NAT) BasedMulti-Homed Routing

As described in U.S. Pat. No. 7,620,037 (“the '037 patent”)—which isincorporated by reference in its entirety, multi-homed network routingappliances can be built that avoid some of the scaling problems posed byBGP and similar traditional Inter AS routing protocols. These aresometimes marketed as II dual WAN firewalls” or II dual WAN” routers.They partially solve the scaling problems by being standalone andavoiding routing protocol peering and coordination with adjacent networkservice providers, and by using Network Address Translation (NAT) wherethe source address on outbound traffic is modified, or the destinationaddress on inbound packets, is modified, in an appropriate way.Practitioners of ordinary skill in the art are familiar with such dualor multi homed NAT routing techniques.

It is also well known that in packet network traffic which encountersNAT routers, firewalls and similar devices, existing sessions can break.Existing sessions which embed IP addresses inside transport protocolheaders and inside the application layer section of packets, or whichmaintain complex state-based on IP addresses, can and do break whenthese IP addresses suddenly change as a result of some routing decisionby a device or by sender or receiver. For example, all existing TCPsessions—which are a function of each endpoint's IP address and portnumber, a total of 4 parameters—do break if a packet shows up and the IPaddress does not match in the session table typically called theTransmission Control Block (TCB); when such an event occurs, end systemsbehaving according to the TCP standard send a TCP reset packet to thesource, which requires the TCP session, and therefore associatedapplication, to terminate abruptly.

In making NAT based routing decisions for reasons of network failure ordegrade, the detection mechanism for such conditions must be highlyreliable, otherwise there will be a serious ‘false positive’ problemwith sessions breaking needlessly.

“Application continuity” and “no disruption of calls or sessions” and“session persistence” are all inter-changeable terms in the backgroundof the invention.

Given that NAT routers which are multiple homed frequently breakexisting sessions when performing a re-route, the problem of a lack ofapplication continuity is a serious one for connected vehicles, as NATwill be expected to play a major role in providing connectivity tomoving vehicles.

1.5 Holes in Geographic Coverage by a Network Service

A very difficult and new problem in mobile in-vehicle LANs is the lackof ubiquitous geographic coverage by any single network service.Typically, single network services—such as LTE operators, and satelliteoperators etc., are run as independent routing domains, i.e. they areseparate Autonomous Systems (ASes) with separate IP address blocksassigned to them for purposes of global routing.

Each individual wireless network service is unable to provide highperformance coverage everywhere today, for a variety of reasonsincluding network congestion that varies in time and place. These resultin “network holes” in the geographic coverage map of a service provider,something which every mobile phone user experiences when a calldisconnects or is unable to complete when traveling down various roads,or while at sea. As used herein, a network hole can refer one or moregeographic regions in which network service connectivity for one or moreproviders is significantly degraded and/or unavailable. Real time, asopposed to static, network degradations can be measured using virtuallyany network connectivity parameter, including but not limited to one ormore of: packet loss rate in each direction, available bit rate in eachdirection, jitter in each direction, round trip time (RTT), one-waylatency, out-of-sequence packet conditions, wireless signal strength,and others, without limitation.

With LTE and 4G/5G networks, two major problems are: (1) the high costof operating cell towers and base stations to provide universalcoverage; and (2) the high cost of backhaul networks to connect eachcell site to the core network of the mobile operator. In the case of thelatter, they are needed in proportionately larger quantities as thetraffic in each cell site grows, and they are costly. Due to theseeconomic factors, the end result is poor or no coverage in many areas,including urban areas which can be so badly congested that each user'sbit rate allocation is reduced.

In satellite networks, rain and moisture in the air can severelyattenuate signals, especially in the higher frequency Ka bands used forhigh throughput satellite services. Lower frequency bands, such as theKu band, are less impacted by rain and moisture.

Many wireless networks (e.g., microwave networks) also require a line ofsight between transmitter and receiver. Line of sight problems can arisedue to tunnels, trees, tall buildings, etc. These also result in holesin network coverage.

A network hole by itself is not a binary concept—aside from no signal or100% signal strength, there are a whole range of potential network pathproblems that can cause packet loss over wireless communications. Theconcept of a network hole is therefore multi-layered—each networkperformance metric such as packet loss rate, available bit rate, jitter,RTT latency—if it crosses certain well known thresholds—can have adversebearing on specific applications.

Such data—whether static data or live data in real time—on network holesand their composite nature are today not widely disseminated by wirelessservice providers to their users, assuming they have the data in thefirst place. It is fair to say it is not easy to collect and update suchinformation easily on a country wide or world wide scale.

1.6. Plain Load Balancing is Insufficient

Some WAN routers perform load balancing of traffic in static ways e.g.per-packet or per-flow, for example, using a round-robin placement amonga set of 2 or more alternate paths. In the context of moving vehiclesand the presence of network holes, this approach does not work anymore,primarily because user sessions can break immediately when a NAT isinvolved, and many in-vehicle embedded applications do not have anoption for manual restart, unlike human interactive applications. A muchmore cognitive and intelligent approach is needed than the traditionalNAT router approach.

DESCRIPTION OF THE VARIOUS EMBODIMENTS

Aspects of the disclosed technology address the foregoing problems ofvehicle connectivity by providing methods and systems for ensuringvehicle network connectivity, for example, by providing systems andmethods for performing application flow placement that anticipatespotential location-specific points of network degradation in ageographic coverage map (network holes). In some aspects, the disclosedtechnology provides systems and methods for identifying network holesalong a specific vehicle path, and for intelligently routing trafficflows over available (alternative) provider paths to avoid networkconnectivity disruptions, i.e., to avoid network holes. Additionally,the disclosed technology provides for predictive flow switching, forexample, by calculating upcoming network holes on a vehicle path and forintelligently predicating provider path availability for current andfuture network flows along the vehicle path.

As used herein, network holes can refer to a loss or degradation ofnetwork connectivity in at a particular geographic point or over ageographic area. In some instances, network holes may refer to a lack ofcoverage (or degraded coverage), with respect to a particular networkprovider path (e.g., with respect to a particular ISP).

FIG. 1A illustrates an example of a network connectivity environmentthat can include wired and/or wireless devices that a moving vehicle islikely to encounter, with varying degrees of connectivity or noconnectivity. Specifically, FIG. 1A shows two moving vehicles 101 and102 in a large zone covered by two wireless network service providers121 and 122. The areas where networks 121 and 122 have adequate wirelesssignal coverage areas are shown as 141 and 143, respectively. By way ofexample, network 121 could be a cellular LTE provider and 122 could be alow earth orbit (LEO) satellite service provider. However, it isunderstood that network 121 can represent any type of data network thatis maintained or administered by any network provider. The variousservices shown in FIG. 1A include an Intelligent Vehicle ConnectivityAnalytics (IVCA) gateway 181, a high definition map server 182 and aweather services server 183. It is understood that the environment ofFIG. 1 could include additional (or fewer) service providers of similar(or different) type, without departing from the scope of the disclosedtechnology.

FIG. 1B conceptually illustrates an example of how network holes canappear as a vehicle moves, and how various servers and services can aidin their timely detection. Specifically, FIG. 1B illustrates how, usingthe methods to be described here, the moving vehicle 101 canautomatically detect a localized network hole 165, specific to aparticular network provider, e.g., wireless provider 121.

FIG. 2 illustrates an example of a Predictive Multi Homing Switch andhow it can be operatively implemented inside existing in-vehicle localarea networks (LANs) and configured to provide continuous applicationconnectivity to a computer network, such as the Internet. Specifically,FIG. 2 shows an example of the key embedded system inside the movingvehicles 101 and 102, the Layer 2-7 Predictive Multi Homed WAN Switch(PMHS) 210. This illustrates one possible embodiment of the invention.

In one aspect, PMHS 31 connects the various LAN segments comprising thevarious in vehicle LANs—which may be both wired LANs and WLANs such as241, 242 and 243—to the larger WAN 200, example connections as in FIG.2. The PMHS 210 performs observed measurements of natural trafficbetween individual vehicle Electronic Control Units (ECUs), TelematicControl Units (TCUs), infotainment units and other devices and externalWAN located devices. The PMHS 210 also performs self-initiatedconnectivity measurements, the active “probes” described in the '037patent, of each ISP access cloud e.g. access clouds 281, 282, 283 and284, in conjunction and with co-operating measurement software runningon an Intelligent Vehicle Connectivity Analytics (IVCA) Gateway 181,which is a measurement server with sufficient computing power tocontinuously take diverse connectivity measurements of large numbers ofmoving vehicles concurrently.

In addition to being LAN connectivity options for PMHS 210, the networkssuch as DSRC 242 and WiFi LAN 241 can also be implemented as WANpathways, in which case they can function as alternate WAN paths to LTEpath 283 or satellite path 281.

The measurement server or IVCA gateway 181 can be physically locatedvirtually anywhere. In some aspects, IVCA gateway 181 is external to thevehicle and connected to the vehicle via the WAN. The types ofmeasurements received and administered by the IVCA gateway can includebut are not limited to: bit rate throughput, packet loss rate, latency,round trip time (RTT), jitter, out-of-sequence events—in each directionof traffic, and for each possible access cloud path that a vehicle mayconnect to. Without limitation, examples of such access cloud pathsinclude WiFi, DSRC, LTE, 5G, Ethernet, and/or other types of networkbuilding blocks.

A geographical connectivity database is computed, by receiving networkinputs from vehicles that have traversed the local region. Networkmeasurement data can be recorded for specific periods of time, such asin the near or distant past, and/or may be combined or aggregated withrecent measurement data for the same region, and/or for an adjacentgeographic region, such as LANs in mobile vehicles 101 and 102 in FIG.1A.

These inputs can include one or more of: (1) Global Positioning System(CPS) or equivalent geographic location coordinates of the place themeasurement was taken, (2) time stamp information as to when themeasurement was taken, (3) weather data obtained from sensors or fromdata feeds provided by cloud based services—in addition to connectivitymeasurement data collected using the distributed measurement algorithmscomputed by PMHS 210 and IVCA gateway 181.

Using all or a portion of the collected measurement data—CPS, WANservice provider path, weather, connectivity metrics—a composite,multi-layered geographic database, with associated map of connectivityis computed. Updates to the geo-connectivity map can be performedperiodically or continually, for example, as new or updated measurementdata is received in real time. The collected contributions of data frommultiple vehicles in the same region around the same time causes thisgeographical connectivity data set to become very large over arelatively short time, enabling highly accurate and statisticallysignificant inferences to be derived, regarding a number of metrics,which include without limitation, data on application traffic flows,micro-flows and their expected durations.

By combining the speed and route (as in the path or route 131 of FIG. 1Aand FIG. 1B) of the vehicle with the geographic connectivity map data,it is possible for the PMHS 210, working independently in many cases, orin conjunction with the IVCA Gateway 181 in other cases, to predictvarious connectivity events that would be adverse to the smoothperformance of applications, and then to perform predictive applicationflow placement by choosing the best available path from a set ofredundant paths, before the vehicle encounters any network holes. Byperforming such flow placement predictively—application continuity ismaximized, and can be 100% if redundant wireless paths are alwaysavailable while the vehicle is moving.

The IVCA gateway 181 could be connected in one embodiment of theinvention at a data center with a path 174 to the public network 191,one instance of which could be the global Internet.

Application Continuity

We describe here methods to detect or predict network degrades reliably,and auto correct computer application traffic flows by placement onalternate ISP paths, in a number of milliseconds, or less, in a timeinterval small enough that human beings, given slow reaction time whichranges from 170 milliseconds to 250 milliseconds for different stimuli,do not notice that something adverse happened with the computerapplication flow. By ‘computer application flow’ we include not justuser computing devices such as laptops and smart phones, but alsoembedded in-car computers, such as ADAS, TCUs and so on. Applicationflows can include any network traffic that is performed by, or onbehalf, of an application or other software routine. Application flowscan include data for, voice, video or other media types.

Fast Detection

The first step is the fast and reliable detection of network relatedproblems. Discussion of network-related problems is described in U.S.Pat. No. 7,620,037, entitled, “Reliable ISP access cloud state detectionmethod and apparatus” which is incorporated by reference in itsentirety. An example of a Multi-Homing System integrated with anAdaptive ISP Access Cloud State Detection apparatus (ACSD) is describedhere. Techniques to reliably detect the failure of an ISP AccessCloud—which could be an LTE or 5G service provider, or a SatelliteInternet service provider—very quickly in typically milliseconds, aredescribed here.

For example, and without limitation, if the maximum of the most recent30 samples of observed round trip times, out of a large data set ofhundreds or thousands of observed round trip times, is T, then a smallinteger multiple k multiplied by T, i.e. kT becomes the time durationthat it takes to detect a failure of a cloud with very high probability.The mathematical proof behind this statistical approach is described inthe '037 patent. The value k can be as small as 3 or 4. If T=5milliseconds, then reliable failure detect can be performed in as littleas 15 milliseconds. If T=20 microseconds, the failure can be detected in60 microseconds.

Both the Predictive Multi Homing Switch PMHS 210 and the measurementserver or Intelligent Vehicle Connectivity Analytics (IVCA) Gateway 181collect large data sets; the IVCA 181 in addition collects all the datafrom multiple vehicles' PMHS measurements, and also derives variousaggregate statistics and analytical conclusions—related to geographicallocation and other data such as weather, loss of line of sight,etc.—that can be computed in the background.

PMHS 210 is designed to be installed in-line with actual traffic, and soreasonably large data set collection is possible in the local PMHS. Thisallows faster inferences in many cases, such as in Voice Over IP (VOiP),as disclosed in U.S. Pat. No. 8,009,554 (the '554 patent) entitled“Method for Multiple Link Quality of Service for Voice and Video OverInternet Protocol,” which is incorporated by reference in its entirety.In most cases an RTP packet in an individual stream will arrive onceevery 20 milliseconds, therefore non-arrival of any RTP packet in thatstream for a period of 3×20=60 milliseconds could be a reliableindicator that the session is about to break, well within the knownhuman reaction time of 170 milliseconds for an audio stimulus.

More generally, the same statistical methods detailed in the '037,patent can be applied to allow the PMHS 210, or the IVCA gateway workingthrough the PMHS 210, to reliably detect the failure of variouscomponents connected to the in-vehicle LAN, e.g. the TCU or the ADAS, orthe individual LAN components, within micro-seconds (in the case of 100Base Tl Ethernet used as an in car or in vehicle backbone LAN) ornano-seconds (in the case of 1000 Base Tl Ethernet used as an in car orin vehicle backbone LAN) instead of milliseconds.

Auto Correction

The second step after reliable detection of a degradation in networkconnectivity is to perform automatic correction. Some of theseprocedures are described in the '554 patent, such as, techniques to moveSIP and RTP based voice and video stream transmissions betweenindependent network paths, without breaking the stream transmission, inthe presence of NAT operations occurring, is described here in detail.

Auto correction is done by computed placement of the application flowson a network path with better performance metrics, as measured andcomputed. In FIG. 2 there are 4 alternative network paths to choose fromspecifically 281, 282, 283 or 284. The computed placement can becurrent—as when a real time break event is detected and an urgentcorrective action is needed—or it can be anticipatory and predicted, asdescribed further below.

PMHS and Predictive Flow Placement

For purposes of this document, the PMHS apparatus described here isimplements the hardware features of the ACSD apparatus of the '037patent, but goes on to add several more that are critical to movingvehicles and predictive flow switching. Two of these features, withoutlimitation, are: (a) a silicon chip for getting precise geographicpositioning data, e.g. CPS, GNSS or equivalent chip 259 in FIG. 2, (b)digital computing techniques to measure or read the speed of a movingvehicle, for example by sending CANbus commands to an in vehicletachometer, as shown via 230 in FIG. 2, and its direction relative toits current position in a geographic map.

FIG. 2 depicts in vehicle LANs, and how the PMHS 210 connects these tothe WAN 200 via multiple WAN paths, via cellular or satellite modems271, 272, etc., which network packets are transmitted and received on.Each vehicle has an in-vehicle Local Area Network (LAN), or multiplesuch LANs, which could be of various types (802.11 WLAN, 802.3 Ethernet,CanBus, and residing on these LANs are one or more computing devices(see FIG. 1B) each running various networked applications, each of whichmay be connected to the wider IP network 291, one embodiment of whichcould be the public Internet, without limitation.

The WAN paths i.e. network service provider (NSP) paths 281, 283, and284 can—and without limitation—belong to independent Autonomous Systems,a concept in inter domain routing well known to practitioners ofordinary skill in the art. Each of these domains have non-overlapping IPaddress spaces, where these IP addresses are either (1) public andglobally reachable, (2) using a private IP address space, or (3) usingIP address blocks used for ad-hoc networks. In the PMHS making a flowplacement decision, and when NAT or PNAT is used as is typically thecase, the source IP address in every packet must be modified to adifferent source IP address belonging to the separate IP address blockassigned to the new NSP.

The computing devices could be various types of Electronic Control Units(ECUs) such as the CANbus ECUs 230 in FIG. 2 It could also be a TCUtelematics control unit 250 or an infotainment computer (e.g. smartphone 221, laptop 222, video player, or the like), more generally knownas an IVI unit 256, or an Advanced Driver Assist System 258 (ADAS)dependent on receiving updates to a High Definition (HD) map in nearreal time from navigation data service providers connected to the WANcloud.

The Telecommunication Control Unit TCU 250 is used for emergency calling(eCall) or breakdown calling (bCall). The predictive flow switchingtechniques described here provide continuity for voice calls originatingor being received by the TCU 250, as all the redundant pathways 281,282, 283 and 284 to the public Internet 191 are made available to it andcontrolled by the PMHS 210.

An integrated car phone 257 via USB tethering, or via Ethernet bus, isalso depicted in this embodiment. An embodiment of the USB car phonecould be in the form of a mechanical smart phone car holder with a builtin USB port which is used as a WAN option for the tethered smartphone,and with WAN routing service for phone 257 being provided by the PMHS210 with its diverse WAN paths 281, 382, 283 and 284.

As mentioned, the PMHS 210 resides in-line with all network traffic,which means all packets flow through it or are visible to it, and PMHS210 observes and stores all flow data in various tables, such as flowtables and RTT tables disclosed in the '037 patent. These methods offlow collection and classification are similar to methods used in sFlowand Netflow, as described in the '037 and '554 patents, and methods oftable creation, insertion and lookup are well known to practitioners ofordinary skill in the art. Every single application flow and micro-flowthat is observed by PMHS 210 as being set up, will find an entry in thePMHS flow table, as soon as the first packet of the flow or micro flowis observed.

By engineering a choice of at least 2 wireless provider paths, e.g. theLTE path 283 and the satellite path 382, via its connection to the PMHS210, and the reliability of the PMHS 210 can be improved from the 99.9%up time published for each network, to 99.9999% up time for mostapplications, as both network paths are rarely likely to fail at thesame time. This reduces the expected down time per year from 8 hours toonly about 31 seconds.

In-band remote management of the PMHS 210 e.g. via SSH or otherprotocol, that is suitably secure, can be done over any of the publicInternet 291 accessible WAN paths 281, 282, 283 and 284. The PMHSalgorithms provide for non-stop continuity of the in-band remotemanagement application, in the same way as for all other applications asdescribed in this invention.

In addition, an Out-of-Band (OOB) management port 212 is also providedon the PMHS 210, connected by an Out-of-Band (OOB) network path—whichcould be Ethernet or CANbus or USB without limitation—to the carmanufacturer provided TCU, which has a low cost 3G or lower category LTEmodem 251 connected to it. The OOB management port is useful if for somereason the PMHS has a software fault and in such cases it can bedesigned to exit gracefully into a ROM monitor program which can onlycommunicate with the OOB port, which is restricted now to only modem 251as a WAN option. The OOB port can be connected to the TCO optionally viaan Ethernet link 252.

Flow table records include all relevant protocol information at the datalink, network, transport, session, presentation and application layers.In addition, they include current path selection information, andentries to assist in NAT (network address translation) and PAT (portaddress translation) flow placement and appropriate packet headermodifications, whenever a decision is made by the PMHS to switch paths.

Application Sensitivity to Network Conditions a Decision Factor in FlowPlacement

Application sensitivity thresholds are stored in a table or data base onboth the PMHS 210 and the IVCA 181. These sensitivity thresholds arewell known to practitioners of ordinary skill in the art, and constitutestatic knowledge that can be used in network device decision making. Anexample of such information, without limitation, and a correspondingtable are shown below.

TABLE 1 Application Sensitivity Thresholds to Network Metrics by FlowType Packet Loss RTT Throughput/Available Rate latency Jitter Bit RateVoice >1% >250 ms >40 ms <32 kbps Email >3% No effect No effect 200 kbps

In this table, the rows are applications or application flows, withoutlimitation, and the columns are various types of network measurementthresholds, which if crossed will result in significant degrade orcomplete failure of the corresponding application flow.

Only a subset of possible entries are shown above in the table, toillustrate the principle, and without limitation. These kinds ofthreshold values are commonly used in network engineering when expertsperform tasks manually, although the exact values used can vary withinreason.

This table is used to predict geographic regions where adverseconditions are likely to be encountered, as a vehicle moves, as it mayfrom time-to-time, encounter a ‘network hole’ in various forms.

To illustrate, consider the voice application and jitter. If multiplevehicles' PMHS units perform the distributed jitter measurements—see 388and 389 in FIG. 3—in conjunction with IVCA Gateway 301, and if, forexample and without limitation, the average or 80 percentile value isgreater than 40 milliseconds for a specific ISP and a specificgeographic location, the IVCA alone can (a) map out the extent of thisnetwork hole for jitter (b) communicate it in advance to the PMHS insidemoving vehicles, after which (c) the PMHS can make a decision whether ornot to predictively move a flow.

FIG. 3 shows how a replicated data center based geo-connectivitydatabase or geo-connectivity map (351 and 352) for each moving vehicle101 or 102 is constructed, in co-operation with replicated IVCA gateways301 and 312), these IVCA gateways being connected to the public Internet301. The IVCA gateways store sets of crowd-sourced globalgeo-connectivity data, such as obtained from moving vehicles 101 and102. As described below, PMHS systems in all moving vehicles (e.g., 101and 102) can perform measurements cooperatively with the IVCA gateways301 and 312.

The terms geo-connectivity database and geo-connectivity map are for thepurposes of this document inter-changeable. The geo-connectivity map isa visual representation of the geo-connectivity database, which is akind of geographic information database.

FIG. 3 further shows the nature of a local database record 361 whichcould have been reported by vehicle 101, and this record can includelocation e.g. CPS coordinates 371, timestamp 372, local weather data373, and an associated set of localized, instantaneous measurements forISP 1 as 374 and for ISP N as 379, where ISPs 1, . . . , N are thewireless or other network service providers encountered by movingvehicle 101 at a specific location and instant in time.

FIG. 3 also shows without limitation, examples of network measurementsat the instant and place for vehicle 101, such as latency 381, packetloss rate (PLR) inbound 382, throughput in bound 383, available bit rate(ABR) inbound 384, PLR outbound 385, throughput outbound 386, ABRoutbound 387, jitter inbound 388 and jitter outbound 389.

Network Holes and Geo-Connectivity Maps:

As discussed above, the need to collect and store in a databasegeographic information which impacts connectivity was described. Thesetransient phenomena encountered by a moving vehicle are what we refer toas network holes. Examples of such are line-of-sight issues where anetwork signal can become impaired, local rain, fog and weatherconditions, location and extent of road tunnels, tall buildings, andother kinds of physical infrastructure without limitation.

A geographic database, similar to geographic databases implemented forroad navigation or maritime navigation, is implemented here as part ofthe invention described here to capture the exact time and location oftransient or permanent network holes.

However, unlike the geographic databases implemented by Google, Appleand all the cellular service companies like ATT and Verizon, thisdatabase has raw data on network metrics experienced from theperspective of the moving vehicle, including (a) CPS or other equivalentcoordinates e.g. GNSS, etc. without limitation (b) time stamp for dateand time to millisecond granularity (c) set of detailed andbi-directional network metrics at the instant in time for each ofmultiple wireless network service access clouds, including bit rates,latencies, loss rates etc., without limitation. Since cellularproviders, if and when they do perform such measurements, do not sharethis data with the public or with their competitors, such measurementdata as collected by the PMHS 210 and the IVCA Gateway 181 enablescomparison of metrics across all available service providers at eachinstant of time.

We will call this record of transient or permanent network holes aGeo-Connectivity Database. The PMHS and the IVCA computing elements willeach implement a Geo Connectivity Database, which will include CPS (orequivalent) coordinates as part of the data.

Multiple vehicles e.g. 101 and 102 in FIG. 1A are moving in a path 101e.g. a road or a maritime or flight path, without limitation. The pathof the moving vehicle could have wireless or other network servicecoverage as shown by networks 121 and 122. For example, network 121could be a cellular based LTE provider network, with coverage areas 141and 143, and 122 could be a broadband satellite service provider withcoverage area 142. Other network types could be 4G, 5G, LoraWAN, IEEE802.11 WLAN, IEEE 802.11 P, and without limitation.

The wireless networks 121 and 122 have connectivity via some backhaulnetwork operated by a city or a commercial service provider via links171 and 172, to a public network 191, to which all vehicles can possiblyconnect, either as a paid or as a free-of-charge service.

Bad weather in the form of fog or rain can also result in network holes,and another aspect of the invention is the ability to incorporatecurrent weather conditions on a route of a moving vehicle 101 or 102 viaa weather data server 183 connected by a link 175 to the public network191.

Tunnels, tall buildings, and even trees can cause certain types ofwireless networks (e.g. satellite, DSRC) to either deteriorate or losesignal altogether, and are another form of network hole that a movingvehicle requiring connectivity can encounter. High Definition mapservers and services, such as server 182 in FIG. 1B, connected by link173 to public network 191, can be used by a moving vehicle 101 to detectthese kinds of network holes well in advance, in another embodiment ofthe invention.

FIG. 1B shows how a network hole 165 can appear in the distance ahead,while a vehicle is moving. Again, this hole is typically only associatedwith a specific network that some of the in-vehicle applications may beusing. The example below illustrates.

Suppose a car with a PMHS 210 and actively communicating with its IVCAGateway 181 is several miles away from a tunnel through a mountain, andthere are several TCP/HTTP web applications already in progress, over aLEO satellite link. It is known by the IVCA Gateway—from the collectivedata sent by earlier cars which traversed the road earlier that LTEsignals work well inside this tunnel. It is also known from earlier datathat the LEO satellite signals are inadequate inside the tunnel and webTCP sessions will break later if nothing is done. In other words, thefact that a ‘network hole’ for TCP based web applications is about toappear, can be now detected and acted upon by the PMHS.

In order to better comprehend the definition of a geo-connectivity map,or geo-connectivity database, we can summarize, without limitation, thefollowing properties:

-   -   a. It describes network holes in various geographic locations on        land, sea and air that will impact computer-networked        application performance inside a moving vehicle in adverse ways    -   b. Network holes can be either transient, as with a temporary        packet loss or stream jitter situation, or adverse weather in an        area like fog; or permanent, as in a deep road tunnel or tall        trees blocking line-of-sight to a satellite    -   c. In the case of transient network holes they require automated        and real time network measurements from the perspective of a        moving vehicle, as computed by the PMHS to discover them and        communicate their presence, location and time-stamp    -   d. A PMHS may communicate these network hole events to another        PMHS in another moving vehicle, or to roadside infrastructure,        or to an IVCA Gateway as described    -   e. Network holes are not restricted to binary UP or DOWN events,        but include the much larger category of degradations in measured        metrics that impact different applications to different degrees        based on application sensitivity thresholds which are well-known        and are used as a part of active flow placement or migration.        Examples of these include available bit rate, latency, jitter,        and others, as elsewhere described in this specification,        without limitation.

Tracking and Reporting Application Flow Life Times

Flows appear and disappear, e.g. a user fires up a Web page using abrowser, and then leaves the page. The PMHS using its flow tables,computes and performs statistical analysis on flow life times—which canbe optimized for a vehicle user's habits. As examples, the PMHS willhave data such as: 100% of all HTTPS flows complete in 15 minutes, 95%of all HTTPS flows complete in 10 minutes, without limitation.

The PMHS is able to compute these flow lifetimes using standard networkpacket observation and timestamping techniques, which are widely known,and as taught in [5] and [11]. Without limitation, a possible segment ofa flow lifetime table computed by a single MHS in a single vehicle isillustrated in Table 2 below, which is a table of application flow typesorganized in percentile categories for the flow lifetime. The large datasets from the crowd sourcing technique—whereby one or more movingvehicles (e.g., 101 and/or 102) contribute data sets—enable thesepercentiles to yield associated threshold times with a correspondingdegree of statistical reliability. Example below: 95% of all email flowsonly live for 30 minutes, implies if a 5% degree of error in flowswitching was allowed, that the threshold L shown in FIG. 4 can be 30minutes. In the same example, if a 0% error rate was mandated this tableshows that for email flows the value of L is raised to a longer, moreconservative value of 80 minutes.

TABLE 2 Flow types and Percentile Lifetimes 100% 95% 80% 50%Web/HTTP/HTTPS 15 mins 10 mins  1 min 10 secs  Email/POP/SMTP 80 mins 30mins 10 mins  3 mins

Predictive Flow Switching by Pro-Active Migration

The IVCA Gateway collects from all cars' PMHS 210 units messages on (1)flows (2) lifetimes as an array of values by percentile. This is done ona periodic basis, so that the IVCA has a statistically large data set offlow life times, across multiple vehicles and their users.

When a network hole appears that adversely impacts HTTPS flows on aspecific WAN path e.g. 282 in FIG. 2, the task of the PMHS 210 is tomigrate all new HTTPS flows early enough from the satellite path 282 tosay the LTE path 283, so that when the car finally reaches the networkhole, there will be no break of those HTTPS sessions.

By way of example, if the PMHS 210 is 15 minutes away from a networkhole, and statistical data on HTTPS flows are up-to-date as the examplein the table above, then 100% of all HTTPS flows which are currentsessions will age out before the hole is reached by the vehicle. All newflows within 15 minutes before reaching the whole would have been placedby pro-active migration on to a service provider path that did not havea network hole.

WCA Gateway and Measurements

The measurement server or IVCA gateway 181 is typically located externalto the vehicle and connected via the WAN. These measurements are shownin FIG. 4 and are well known to practitioners of ordinary skill in theart. They include but are not limited to: throughput as in 383 and 386,packet loss rate as in 382 and 385, round trip time (RTT) as in 381,jitter as in 388 and 389, in each direction of traffic, and for each ofthe available access cloud paths shown in FIG. 2 e.g. 281, 282, 283,284. Other measurements could include (a) whether the service causespackets to get out of sequence, and/or (b) whether the service isenforcing ingress source IP address filtering, etc.

Role of IVCA Gateway in Creating and Updating Connectivity Maps

Each vehicle's PMHS unit is associated with exactly one IVCA Gateway atinitialization or install time, with redundant IVCA Gateways existingfor fault tolerance, as is well known to practitioners or ordinary skillin the art.

It is important to distinguish between two kinds of network holes:static or permanent, and dynamic or transient. For example, a Line ofSight hole due to a fixed mountain tunnel is static. An example ofdynamic holes are those caused by changing weather, or by changingcongestion conditions in a specific network. The attribute of dynamic orstatic is also stored in the Geo-Connectivity database.

FIG. 4 illustrates the basic method, without limitation, for predictingthe latest time predictive flow switching should be performed in orderto avoid session break caused by an approaching network hole along aroute 432 that a vehicle is navigating on. The key role played by theconstraint equation 450 (strictly an inequality) is explained below.

FIG. 4 also shows one embodiment of the invention where a constraintequation 450 based on the expected vehicle route 432, see FIG. 4, can beused to compute when the latest time can be to start moving applicationflows of a specific sensitivity type.

Predictive Flow Switching and Static Geo-Connectivity Map Data

Most vehicles are parked and stationary for large periods of time.During these periods, the IVCA Gateway will communicate with the PMHSand load or update the entire static connectivity map data, for ageographic region of reasonable size, into the PMHS. When the vehicleeventually begins a journey, and moves, it has all static data storedand is relatively self-reliant in making connectivity decisions forvarious application flows. As described in the '037 patent, the PMHS 210can have a Control Engine that performs computations in thebackground—using CPS as in 1590 in FIG. 2, the CPS coordinates and theexpected speed of the vehicle, it can compute the distance from thepresent location to the location of any network hole, such as hole 401in FIG. 4, and the time expected to reach the network hole 401. Then, ifany set of flows need to be migrated to a different network, the PMHS210 has enough time to migrate application flows before the network holeis reached.

Predictive Flow Switching and Dynamic Geo-Connectivity Map Data

Dynamic connectivity metric changes can occur due to new Line of SightObstructions, or the removal of previous Line of Sight obstructions, theweather e.g. precipitation and moisture, or network related congestionwhich causes high levels of packet loss or jitter.

The IVCA Gateway updates each vehicle's PMHS with dynamic connectivitymap changes at pre-configured or event-driven intervals. Onlysignificant statistical changes detected by the IVCA Gateway 181 need tobe communicated, with the gateway 181 computing a dynamic connectivitymap update in the local region being approached, or expected to beapproached, by the vehicle. In FIG. 4, the “local region beingapproached” is a superset in area of the expected route 432 and thenetwork hole 165. The IVCA Gateway—by virtue of its large data setcollected from many vehicles—and also by access to geographic databasesprovided by other services—can compute the dynamic connectivity mapupdate using a simple procedure: (1) sort all known network holes bydistance from the vehicle to be updated, and eliminate all the holesthat are too far away i.e. so far away that all flows will age out bythe time the vehicle reaches the specific hole, and the basic approachof constraint equation 450 in FIG. 4, along with estimates of maximumdistances traversed for reasonable flow lifetimes L is used in such stepof elimination (2) if there are holes remaining after elimination—thePMHS 210 forms the connectivity map update message as an update to aprevious static connectivity map, and transmits the update message tothe vehicle using one of the available network paths.

Avoiding Breakage of all TCP Sessions

In the specific case of any TCP session, this kind of predictiveapplication placement—for new TCP flows only—avoids having to break TCPsessions in mid-stream, which is what a sudden NAT reroute can cause ifa TCP session is already in process and application layer data isflowing. The concept however is not limited to TCP sessions where simpleobservation of a TCP SYN 3-way handshake signals a new flow.

Put another way, new flow detection for TCP is based on detection of aSYN-SYN-ACK sequence. The SYN can be placed on a network that does notsuffer from the network performance hole. This, without limitation, isan example of pro-active flow migration used to achieve predictive flowswitching.

Avoiding Breakage of Non-TCP Sessions

More generally-whether it is UDP, TCP, IPSEC, GRE or other flows basedon the “IP Protocol” field in the IP header, or “Next Header” field incase of IP version 6, there are early stages in session set up which thePMHS 210 observes, and for which a similar predictive placement decisioncan be made before the actual user session establishes. Examples of suchtriggers for new flow detection include, without limitation (a) a DNSrequest packet searching for the IP address of a specific named resource(b) a Transport Layer Security (TLS) Client Hello packet (c) a SIP/SDPpacket.

Concurrent Application Flow Placement on Multiple WAN Paths

It is important to stress that the large data sets of connectivitymeasurements on both the PMHS acting standalone, and the PMHS acting incooperation with an IVCA Gateway, enable not just intelligent placementwhen faults are about to occur. Every time a new flow appears, the PMHShas a full control on which WAN path to place it on, and traffic willend up being load balanced, with the best WAN path being chosen for eachapplication flow. When both the LTE 283 and Satellite paths 282 areavailable, for example, latency sensitive voice traffic can be placed on283 and throughput sensitive file download or video streaming can beplaced on 282, so that all paths are in use concurrently, and in a waythat maximizes the application performance.

Method for Predictive Flow Switching

One example of a process performed by PMHS 210 in illustrated in FIG. 2.To sum up what has been described, the steps and procedures implementedby the PMHS to accomplish predictive routing which minimizes theprobability of sessions breaking or applications not experiencingcontinuity are shown in FIG. 5, and described here.

FIG. 5 illustrates an example process that includes 9 steps (i.e., 502,504, 506, 508, 510, 520, 530, 540, 550 and 560), that can be performedby the PMHS, in one embodiment of the invention, to achieve localcomputation of the geo-connectivity database 506 and map in itsimmediate vicinity, or radius of interest, based on Table 2 and in-depthpercentile based knowledge of flow lifetimes.

Without limitation, these steps are: (1) PMHS sends request 502 to PMHSForwarding Process for application flow measurements data (local networkmeasurement data) obtained locally without sending requests to an IVCAgateway (in-line measurements); (2) PMHS receives response 504 to the502 request and updates a local geo-connectivity database 506, using thereceived measurement data, when response is received; (3) PMHS sendsrequest 508 to a remotely located IVCA Gateway for geo-connectivity dataof its local region, specifying an approaching area by radius R, whichcould be equal to speed multiplied by L, the percentile flow lifetimeselected for all flows of interest, or some other formula embodying theidea of a radius of interest being proportional to the lifetime of aflow and the velocity of the vehicle relative to its current position,without limitation; (4) PMHS updates local geo-connectivity database 506when response 510 is received from IVCA Gateway; (5) PMHS sends networkmeasurement to IVCA gateway: For each of a plurality of serviceproviders, it requests, to collect metrics e.g. 381 to 389 as in FIG. 3,and appends the vehicle's identifier, location and timestamp withrequest. Measurements are then performed and stored at both PMHS andIVCA gateway, as in step 520; (6) PMHS computes presence of approachingnetwork holes H, for each of a plurality of provider network paths P, byanalyzing data in the geo-connectivity database, as in step 530; (7)PMHS computes the set of application flow types A(H,P) that will beadversely impacted by next approaching network hole H, if placed on afirst provider network path P as in 540; (8) PMHS Control processcomputes the clock time T after which all new flows of type found in theset A(H,P) will be placed on a provider path other than P, and also thebest path Q other than P, taking into account the speed and direction ofthe vehicle as in 550; (9) When the clock time reaches T, any new flowsdetected by the PMHS and which is of a type that belongs to set A(H,P)will be placed on the best alternate path Q instead of P as in 560.

Summary of IVCA Procedures to Build a Statistically Significant Data Seton Geo-Connectivity Across all Geographies

One embodiment of the IVCA gateway is 312 as shown in FIG. 3. In FIG. 3two IVCA gateways are shown for redundancy and fault tolerance, and inthis embodiment are hosted in a data center with redundant connectionsto the global Internet.

FIG. 6 shows how an example of how the IVCA starts with a geographicinformation system (GIS) database 604 and uses crowd-sourced networkmetrics obtained from moving vehicles to compute a geo-connectivitydatabase 610, and then co-operatively assists vehicles on request, toperform predictive flow switching.

Without limitation, a summary of steps taken by the IVCA to accomplishpredictive flow switching are:

In step 602, IVCA Gateway initializes geographic information systemdatabase 604 for constructing global maps and navigation routes for allkinds of vehicles. As discussed above, vehicles can include, withoutlimitation, cars, motorcycles, trains, ships, drones, and planes, etc.).

In step 606, IVCA Gateway waits for requests or updates from a one ormore moving vehicles, or from various geographic information services.Updates can include information on factors affecting networkconnectivity (e.g., network measurements or measurement data), suchinformation pertaining to tunnels, weather, positions of LEO satellites,or other factors that can affect where interference might arise with GSOsatellites, etc.

In step 608, IVCA processes a network measurement request from a movingcar's PMHS: For each of a plurality of service providers, it conductsmeasurements in conjunction with the PMHS, collecting metrics e.g. 381to 389 as in FIG. 3, appends the vehicle's identifier, location andtimestamp, and creates a Global Geo Connectivity Database 610.

In step 620, the IVCA Gateway aggregates data from plurality of vehiclesand computes statistics and analytics data, then appends thesestatistics and analytics data into database 610.

In step 630, the IVCA Gateway computes network holes and organizes thesenetwork holes by region local to any specific vehicle and updatesdatabase 610.

In step 640, IVCA Gateway processes update packet from a moving car PMHSwhich detected a new network hole problem on a route, using the PMHS ownin-line network path measurements, and IVCA updates database 610.

Alternative Network Paths Using DSRC and Vehicle-to-Vehicle

FIG. 7 illustrates a very useful embodiment of the invention wheretunnels which block LTE signals are involved, and the PMHS units inmultiple vehicles are able to form an alternate path successfully, whereCar A, connected to a first provider path as in 702, can effectively berouted via Car C to a second provider path as in 704. Note also that CarA and Car C can be connected to different LTE service providers.

In FIG. 7, second path 704 is made possible by the PMHS using its DSRCinterface to compute a Vehicle to Vehicle path, through the tunnel,finally reaching a vehicle C which has a path 706 via LTE to the publicInternet.

Alternative Provider Path via LTE Enabled Tunnels

FIG. 8 describes another embodiment of the invention, where the geoconnectivity map at the IVCA gateway has been updated with a newtopological discovery that the specific tunnels in both examples 820 and840 are LTE enabled. This information can be obtained either via ageographical information service, or from measurements provided byvehicle PMHS units in their periodic updates to the IVCA, which the IVCAsends to the PMHS in any vehicle when it makes a request for localregion network hole data.

In Step 810: Before cars reach these tunnels, a connection is onSatellite link 802. These tunnels are known to the IVCA Gateway to beLTE enabled inside. The PMHS learns this information ahead of time fromthe IVCA Gateway. Predictive Flow Switching in the PMHS moves theconnection automatically from Satellite link 802 to LTE links 804, 806and 808 prior to entering tunnel to ensure application continuity and nocall dropped.

FIG. 9 illustrates an example of an electronic device with which someaspects of the technology can be implemented. Specifically, FIG. 9illustrates various hardware components that could be used to implementeither an IVCA or PMHS of the subject technology.

Device 900 includes a master central processing unit (CPU) 962,interfaces 968 (e.g., network interfaces), and a bus 915 (e.g., a PCIbus). When acting under the control of appropriate software or firmware,CPU 962 can be configured to perform operations necessary to collect,store and/or transmit measurement data, including information describingconnectivity metrics for at least one network provider (e.g., whenconfigured to function as a PMHS). Alternatively, device 900 may beconfigured to function as an IVCA gateway, as discussed above, in whichcase CPU 962 can be configured to aggregate measurement data from aplurality of moving vehicles communicating and conducting networkmeasurements in real time (e.g., describing geo-connectivity for variousnetwork providers), and to manage and update a geo-connectivity databasebased on the received measurement data. CPU 962 can accomplish thesefunctions under the control of software including an operating systemand other applications software. CPU 962 can include one or moreprocessors 963 such as a processor from the Motorola family ofmicroprocessors, the ARM family of microprocessors, or the MIPS familyof microprocessors. In an alternative embodiment, processor 963 isspecially designed hardware for controlling the operations of device911. In a specific embodiment, a memory 961 (such as non-volatile RAMand/or ROM) also forms part of CPU 962. However, there are manydifferent ways in which memory could be coupled to the system.

The interfaces 968 can be provided as interface cards (sometimesreferred to as “line cards”), or wireless interfaces, for example, thatare configured to provide wireless network access using one or more RFtransceivers. Interfaces 968 control the sending and receiving of datapackets over the network and sometimes support other peripherals usedwith a router. Among the interfaces that can be provided are, withoutlimitation, gigabit Ethernet interfaces, frame relay interfaces, cableinterfaces, Digital Subscriber Line (DSL) interfaces, token ringinterfaces, WiFi IEEE 802.11a/b/g/n wireless interfaces, DSRC IEEE802.11p wireless interfaces, Geo Stationary Satellite (GSO) interfaces,Low Earth Orbit (LEO) Satellite interfaces, and the like. Generally,these interfaces may include ports appropriate for communication withthe appropriate media. In some cases, they may also include anindependent processor and, in some instances, volatile RAM. Theindependent processors may control such communications intensive tasksas packet switching, media control and management. By providing separateprocessors for the communications intensive tasks, these interfacesallow the master microprocessor 962 to efficiently perform routingcomputations, network diagnostics, security functions, etc.

Although the device shown in FIG. 9 is one example device of the presentinvention, it is by no means the only network device architecture onwhich the present invention can be implemented. For example, anarchitecture having a single processor that handles communications aswell as routing computations, etc. is often used. Further, other typesof interfaces and media could also be used with the router.

Regardless of the network device's configuration, it may employ one ormore memories or memory modules (including memory 961) configured tostore program instructions for the general-purpose network operationsand mechanisms for roaming, route optimization and routing functionsdescribed herein. The program instructions may control the operation ofan operating system, the operations required of a network control planeprocessor, the operations required of a network forward plane processor,and/or one or more applications, for example. The memory or memories mayalso be configured to store tables such as application microflow tables,flow lifetime tables, application sensitivity threshold tables, mobilitybinding, registration, and association tables, and other data pertinentto detection and prediction of network holes.

Although the exemplary embodiment described herein employs storagedevice 961, it should be appreciated by those skilled in the art thatother types of computer readable media which can store data that areaccessible by a computer, such as magnetic cassettes, flash memorycards, digital versatile disks, cartridges, random access memories(RAMs), read only memory (ROM), a cable or wireless signal containing abit stream and the like, may also be used in the exemplary operatingenvironment. Non-transitory computer-readable storage media expresslyexclude media such as energy, carrier signals, electromagnetic waves,and transitory signals per se.

To enable user interaction with the computing device 900, interfaces 968can represents any number of input mechanisms, such as a microphone forspeech, a touch-sensitive screen for gesture or graphical input,keyboard, mouse, motion input, speech and so forth. Interfaces 968 canalso represent one or more output devices or mechanisms known to thoseof skill in the art. In some instances, multimodal systems enable a userto provide multiple types of input to communicate with device 900.

For clarity of explanation, the illustrative system embodiment ispresented as including individual functional blocks including functionalblocks labeled as a “processor” or processor 963. The functions theseblocks represent may be provided through the use of either shared ordedicated hardware, including, but not limited to, hardware capable ofexecuting software and hardware, such as a processor 963, that ispurpose-built to operate as an equivalent to software executing on ageneral purpose processor. For example, the functions of one or moreprocessors may be provided by a single shared processor or multipleprocessors. (Use of the term “processor” should not be construed torefer exclusively to hardware capable of executing software.)Illustrative embodiments may include microprocessor and/or digitalsignal processor (DSP) hardware, read-only memory (ROM) for storingsoftware performing the operations discussed below, and random accessmemory (RAM) for storing results. Very large scale integration (VLSI)hardware embodiments, as well as custom VLSI circuitry in combinationwith a general purpose DSP circuit, may also be provided.

The logical operations of the various embodiments are implemented as:(1) a sequence of computer implemented steps, operations, or proceduresrunning on a programmable circuit within a general use computer; (2) asequence of computer implemented steps, operations, or proceduresrunning on a specific-use programmable circuit; and/or (3)interconnected machine modules or program engines within theprogrammable circuits. The system 968 can practice all or part of therecited methods, can be a part of the recited systems, and/or canoperate according to instructions in the recited non-transitorycomputer-readable storage media. Such logical operations can beimplemented as modules configured to control the CPU 962 or processor963 can be configured to perform particular functions according to theprogramming of the module. By way of example, one or more processors canbe configured to execute operations including: receiving measurementdata comprising one or more connectivity metrics for at least onenetwork provider, and updating a geo-connectivity database using thereceived measurement data.

In some aspects, processor 963 can be further configured to performoperations for transmitting a geo-connectivity request to an IntelligentVehicle Connectivity Analytics (IVCA) gateway, and receiving ageo-connectivity reply from an IVCA gateway, the geo-connectivity replycomprising information regarding network availability for the at leastone network provider along a vehicle path. In some aspects, processor963 may be configured to perform operations for computing a presence ofone or more network holes on the vehicle path based on thegeo-connectivity reply, and identifying a threshold time, after which,new network traffic flows are directed to an available provider path,based on the one or more network holes on the vehicle path. In someaspects, processor 963 may be configured to perform operations forcomputing a presence of one or more network holes on the vehicle pathbased on the geo-connectivity reply; and identifying one or moreexisting network traffic flows for migration to a different providerpath, based on the one or more network holes. As discussed above, thethreshold time can be based on one or more of: the vehicle path, avehicle direction, or a vehicle speed. Additionally, computing thepresence of one or more network holes can be performed for each of aplurality of provider paths.

It is understood that any specific order or hierarchy of steps in theprocesses disclosed is an illustration of exemplary approaches. Basedupon design preferences, it is understood that the specific order orhierarchy of steps in the processes may be rearranged, or that only aportion of the illustrated steps be performed. Some of the steps may beperformed simultaneously. For example, in certain circumstances,multitasking and parallel processing may be advantageous. Moreover, theseparation of various system components in the embodiments describedabove should not be understood as requiring such separation in allembodiments, and it should be understood that the described programcomponents and systems can generally be integrated together in a singlesoftware product or packaged into multiple software products.

The previous description is provided to enable any person skilled in theart to practice the various aspects described herein. Variousmodifications to these aspects will be readily apparent to those skilledin the art, and the generic principles defined herein may be applied toother aspects. Thus, the claims are not intended to be limited to theaspects shown herein, but are to be accorded the full scope consistentwith the language claims, wherein reference to an element in thesingular is not intended to mean “one and only one” unless specificallyso stated, but rather “one or more.”

A phrase such as an “aspect” does not imply that such aspect isessential to the subject technology or that such aspect applies to allconfigurations of the subject technology. A disclosure relating to anaspect may apply to all configurations, or one or more configurations. Aphrase such as an aspect may refer to one or more aspects and viceversa. A phrase such as a “configuration” does not imply that suchconfiguration is essential to the subject technology or that suchconfiguration applies to all configurations of the subject technology. Adisclosure relating to a configuration may apply to all configurations,or one or more configurations. A phrase such as a configuration mayrefer to one or more configurations and vice versa.

The word “exemplary” is used herein to mean “serving as an example orillustration.” Any aspect or design described herein as II exemplary” isnot necessarily to be construed as preferred or advantageous over otheraspects or designs.

1. A multi-homing system, comprising: a network interface coupled to oneor more processors, the network interface configured for communicationwith at least one computer network; and a computer-readable mediumcoupled to the one or more processors, the computer-readable mediumcomprising computer instructions stored therein, which when executed bythe one or more processors, cause the one or more processors to performoperations comprising: determining a first location of the multi-homingsystem; computing first connectivity metrics for a first networkprovider at the first location; computing second connectivity metricsfor a second network provider at the first location; computing a firstpath routing decision based on the first connectivity metrics and thesecond connectivity metrics; updating a locally stored geo-connectivitydatabase based on the on first connectivity metrics and the secondconnectivity metrics; determine a second location of the multi-homingsystem; computing updated first connectivity metrics for the firstnetwork provider at the second location; computing updated secondconnectivity metrics for the second network provider at the secondlocation; and selecting a preferred network provider at the secondlocation for one or more application flows associated with themulti-homing system, based on the updated first connectivity metrics andthe updated second connectivity metrics; and moving an application flowfrom the first network provider to the second network provider withoutdisrupting the application continuity experience, wherein theapplication flow comprises one or more of audio data and video data, andwherein the application flow uses one or more of a session initiationprotocol (SIP) or a real-time transport protocol (RTP).
 2. Themulti-homing system of claim 1, wherein the application flow comprisesaudio data and wherein the application flow uses SIP.
 3. Themulti-homing system of claim 1, wherein the application flow comprisesvideo data and wherein the application flow uses SIP.
 4. Themulti-homing system of claim 1, wherein the application flow comprisesaudio data and wherein the application flow uses RTP.
 5. Themulti-homing system of claim 1, wherein the application flow comprisesvideo data and wherein the application flow uses RTP.
 6. Themulti-homing system of claim 1, wherein the one or more processors arefurther configured to perform further operations, comprising: comparingthe updated first connectivity metrics for the first network providerwith the updated second connectivity metrics for the second networkprovider to identify the preferred network provider at the secondlocation of the multi-homing system.
 7. The multi-homing system of claim1, wherein the one or more processors are further configured to performfurther operations, comprising: predicting a network connectivity pathat a third location based on a route of the multi-homing system.
 8. Themulti-homing system of claim 1, wherein the one or more processors arefurther configured to perform further operations, comprising: updatingthe locally stored geo-connectivity database based on the updated firstconnectivity metrics, the updated second connectivity metrics, and thesecond location.
 9. The multi-homing system of claim 1, wherein themulti-homing system is located within a vehicle.
 10. A non-transitorycomputer-readable storage medium including instructions that, whenexecuted by one or more processors, cause the one or more processors toperform operations comprising: determining a first location of amulti-homing system; computing first connectivity metrics for a firstnetwork provider at the first location; computing second connectivitymetrics for a second network provider at the first location; computing afirst path routing decision based on the first connectivity metrics andthe second connectivity metrics; updating a locally storedgeo-connectivity database based on the on first connectivity metrics andthe second connectivity metrics; determine a second location of themulti-homing system; computing updated first connectivity metrics forthe first network provider at the second location; computing updatedsecond connectivity metrics for the second network provider at thesecond location; and selecting a preferred network provider at thesecond location for one or more application flows associated with themulti-homing system, based on the updated first connectivity metrics andthe updated second connectivity metrics; and moving an application flowfrom the first network provider to the second network provider withoutdisrupting the application continuity experience, wherein theapplication flow comprises one or more of audio data and video data, andwherein the application flow uses one or more of a session initiationprotocol (SIP) or a real-time transport protocol (RTP).
 11. Thenon-transitory computer-readable storage medium of claim 16, wherein theapplication flow comprises audio data and wherein the application flowuses SIP.
 12. The non-transitory computer-readable storage medium ofclaim 16, wherein the application flow comprises video data and whereinthe application flow uses SIP.
 13. The non-transitory computer-readablestorage medium of claim 16, wherein the application flow comprises audiodata and wherein the application flow uses RTP.
 14. The non-transitorycomputer-readable storage medium of claim 16, wherein the applicationflow comprises video data and wherein the application flow uses RTP. 15.The non-transitory computer-readable storage medium of claim 16, whereinthe multi-homing system is located within a vehicle.
 16. A method,comprising: determining a first location of a multi-homing system;computing first connectivity metrics for a first network provider at thefirst location; computing second connectivity metrics for a secondnetwork provider at the first location; computing a first path routingdecision based on the first connectivity metrics and the secondconnectivity metrics; updating a locally stored geo-connectivitydatabase based on the on first connectivity metrics and the secondconnectivity metrics; determine a second location of the multi-homingsystem; computing updated first connectivity metrics for the firstnetwork provider at the second location; computing updated secondconnectivity metrics for the second network provider at the secondlocation; and selecting a preferred network provider at the secondlocation for one or more application flows associated with themulti-homing system, based on the updated first connectivity metrics andthe updated second connectivity metrics; and moving an application flowfrom the first network provider to the second network provider withoutdisrupting the application continuity experience, wherein theapplication flow comprises one or more of audio data and video data, andwherein the application flow uses one or more of a session initiationprotocol (SIP) or a real-time transport protocol (RTP).
 17. The methodof claim 16, wherein the application flow comprises audio data andwherein the application flow uses SIP.
 18. The method of claim 16,wherein the application flow comprises video data and wherein theapplication flow uses SIP.
 19. The method of claim 16, wherein theapplication flow comprises audio data and wherein the application flowuses RTP.
 20. The method of claim 16, wherein the application flowcomprises video data and wherein the application flow uses RTP.